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Complex biological forms reproduce by taking advantage of an arbitrarily 
complex set of auto-catalyzing chemical reactions. Biological life is in control of its 
own means of reproduction, and this autonomy of design and manufacture is a key 
element which has not yet been understood or reproduced artificially. To this date, 
robots — a form of artificial life'— are still designed laboriously and constructed by 
teams of human engineers at great cost. Few robots are available because these costs 
must be absorbed through mass production that is justified only for toys, weapons, 
and industrial systems like automatic teller machines. 

Here we report a set of experiments in which simple electro-mechanical systems 
evolve from scratch to yield physical locomoting machines. Like biological lifeforms 
whose structure and function exploit the behaviors afforded by their own chemical 
and mechanical medium, our evolved creatures take advantage of the nature of 
their own medium - thermoplastic, motors, and artificial neurons’. We thus achieve 
autonomy of design and construction using evolution in a limited universe physical 
simulation** coupled to off-the-shelf rapid manufacturing technology’. This is the 
first time robots have been robotically designed and robotically fabricated. 

The field of Artificial Life examines “life as it could be” based on understanding 
the principles and simulating the mechanisms of real biological forms’. Just as airplanes 
use the same principles as birds, but have fixed wings, artificial lifeforms may share the 
same principles, but not the same implementation in chemistry. Every feature of living 
systems seems wondrous until it is understood: Stored energy, autonomous movement, 
and even animal communication are no longer miracles, as they are replicated in toys 
using batteries, motors, and computer chips. 

Our key claim is that to realize artificial life, full autonomy must be attained not 
only at the level of power and behavior (the goal of robotics, today’), but also at the 


levels of design and fabrication. Only then can we expect synthetic creatures to bootstrap 


and sustain their own evolution. We thus seek automatically designed and constructed 
physical artifacts that are (a) functional in the real world, (b) diverse in architecture 
(possibly each slightly different), and (c) producible in short turn-around time, low cost 
and large quantities. So far these requirements have not been met’. 

The experiments described here use evolutionary computation for design, and 
additive fabrication for reproduction. The evolutionary process operates on a population 
of candidate robots, each composed of some repertoire of building blocks. The 
evolutionary process iteratively selects fitter machines, creates offspring by adding, 
modifying and removing building blocks using a set of operators, and replaces them into 
the population (see methods section). Evolutionary computation has been applied to many 
engineering problems? "®. However, studies in the field of evolutionary robotics reported 
to date involve either entirely virtual worlds?4, or, when applied in reality, adaptation of 


11,12,13 These robots 


only the control level of manually designed and constructed robots 
have a predominantly fixed architecture, although Lund'* evolved partial aspects of the 
morphology, Thompson’? evolved physical electric circuits for control only, and we 
evolved static Lego structures, but had to manually construct the resultant designs'®. 
Other works involving real robots make use of high-level building blocks comprising 
significant pre-programmed knowledge'’. Similarly, additive fabrication technology has 
been developing in terms of materials and mechanical fidelity'® but has not been placed 
under the control of an evolutionary process. 

Our approach is based on use of only elementary building blocks and operators in 
both the design and fabrication process. As building blocks are more elementary, any 
inductive bias associated with them is minimized, and at the same time architectural 
flexibility is maximized. Similarly, use of elementary building blocks in the fabrication 
process allows it to be more systematic and versatile. As a theoretic extreme, if we could 
use only atoms as building blocks, laws of physics as constraints and nano-manipulation 
for fabrication, the versatility of the design space would be maximized. Earlier reported 
work used higher-level components and limited architectures (like only tree structures? ^) 
resulted in expedited convergence to acceptable solutions, but at the expense of 
truncating the design space. Furthermore, these design spaces did not consider 


manufacturability. 


The design space we used was comprised of bars and actuators as building blocks 
of structure and artificial neurons as building blocks of control. Bars connected with free 
joints can potentially form trusses that represent arbitrary rigid, flexible and articulated 
structures as well as multiple detached structures, and emulate revolute, linear and planar 
joints at various levels of hierarchy. Similarly, sigmoidal neurons can connect to create 
arbitrary control architectures such as feed-forward and recurrent nets, state machines and 
multiple independent controllers (like multiple ganglia). Additive fabrication, where 
structure is generated layer by layer, allows automatic generation of arbitrarily complex 
physical structures and series of physically different bodies, including any composed of 
our building blocks. A schematic illustration of a possible architecture is shown in 1. The 
bars connect to each other through ball-and-socket joints, neurons can connect to other 


neurons through synaptic connections, and neurons can connect to bars. In the latter case, 
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Figure 1. Schematic illustration of an evolvable robot Bars connect to each other to form arbitrary trusses; by 
changing the number of bars and the way they connect, the structural behavior of the truss is modified: some 
substructures may become rigid, while others may become articulated. Neurons connect to each other via synapses to 
form arbitrary recurrent neural networks. By changing the synapse weights and activation threshold of the neuron, the 
behavior of the neuron is modified. By changing the number of neurons and their connectivity, the behavior of the 
network is modified. Also, we allow neurons to connect to bars: in the same way that a real neuron governs the 
contraction of muscle tissue, the artificial neuron signal will control the length of the bar by means of a linear actuator. 
All these changes can be brought about by mutational operators. A sequence of operators will construct a robot and its 
controller from scratch by adding, modifying and removing building blocks. The sequence at the bottom of the image 
illustrates an arbitrary progression of operators that create a small bar, elongate it, and split it. Simultaneously, other 
operators create a neuron, add another neuron, connect them in a loop, and eventually connect one of the neurons to 
one of the bars. The bar is now an actuator. Since no sensors were used, these robots can only generate patterns and 
actions, but cannot directly react to their environment. 


€ Ancestral Proximity > € Ancestral Proximity > 


BKC). = 


€ Ancestral Proximity > € Ancestral Proximity > 
(c) (d) 

Figure 2. Phylogenetic trees of several different evolutionary runs. Each node in the tree represents 
an individual and links represent parent-child relationship. Vertical axis represents generations and 
horizontal axis represents ancestral proximity in terms of hops along the tree necessary to get from 
one individual to another. All trees originate at a common root denoting an empty robot with zero 
bars and actuators. Trees exhibit various degrees of divergence and speciation: (a) extreme 
divergence, resulting from niching methods?” (b) extreme convergence, resulting from fitness- 
proportionate selection (c) intermediate level of divergence, typical of earlier stages of fitness- 
proportionate selection and (d) massive extinction under fitness proportionate selection. The trees 
were thinned and depict several hundred generations each. 


the length of the bar is governed by the output of the neuron, by means of a linear 
actuator. No sensors were used. 

Starting with a population of 200 machines that were comprised initially of zero 
bars and zero neurons, we conducted evolution in simulation. The fitness of a machine 
was determined by its locomotion ability: the net distance its center of mass moved on an 
infinite plane in a fixed duration. The process iteratively selected fitter machines, created 
offspring by adding, modifying and removing building blocks, and replaced them into the 


population (see methods section). This process typically continued for 300 to 600 
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Figure 3. A generation. An arbitrarily sampled instance of an entire generation, thinned down 
to show only significantly different individuals. The caption under each image provides an arbitrary 
index (used for reference) and the fitness of that individual. Two subpopulations of robots are 
observable, each with its own variations: one flat on the ground, and the other containing some elevated 
structure. 


generations. Both body (morphology) and brain (control) were thus co-evolved 
simultaneously. 

The simulator we used for evaluating fitness (see methods section) supported 
quasi-static motion in which each frame is statically stable. This kind of motion is 
simpler to transfer reliably into reality, yet is rich enough to support low-momentum 
locomotion. Typically, several tens of generations passed before the first movement 
occurred. For example, at a minimum, a neural network generating varying output must 
assemble and connect to an actuator for any motion at all (see sequence in Fig 1 for an 
example). Various patterns of evolutionary dynamics emerged, some of which are 


reminiscent of natural phylogenic trees. Figure 2 presents examples of extreme cases of 


convergence, speciation, and massive extinction. A sample instance of an entire 
generation, thinned down to unique individuals is shown in Figure 3. 

Selected robots out of those with winning performance were then automatically 
replicated into reality: their bodies, which exist only as points and lines, were first 
converted into a solid model with ball-joints and accommodations for linear motors 
according to the evolved design (Fig 4a). This solidifying stage was performed by a 
automatic program which combined pre-designed components describing a generic bar, 
ball joint, and actuator. The virtual solid bodies were then materialized using commercial 
rapid prototyping technology (Fig 4b). This machine used a temperature-controlled head 
to extrude thermoplastic material layer by layer, so that the arbitrarily evolved 
morphology emerged as a solid three-dimensional structure without tooling or human 


intervention. The entire pre-assembled machine was printed as a single unit, with fine 
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Figure 4. Physical embodiment process: (a) Automatically fleshed joints in virtual space, 
(b) physical replication process in a rapid prototyping machine that build the three dimensional 
morphology layer after layer, (c) pre-assembled body in mid print, (d) a close-up image of a joint 
printed as a single unit. Note ball is printed inside the socket. 


Table I: Results 


Distance traveled [cm] Virtual Physical 


Tetrahedron (Figure 5a) 38.5 38.4 (35) 
Arrow (Figure 5b) 59.6 22.5 (18) 


Pusher (Figure 5c) 85.1 23.4 (15) 


Comparison of performance of physical creatures versus their virtual origin. Values are net 
distance [cm] center of mass traveled over 12 cycles of neural network. Distances in physical column 
are compensated for scale reduction (actual distance in parentheses). Mismatch in last two rows is 
primarily due to slipping of limbs on surface. 


plastic supports connecting between moving parts (Fig 4c); these supports broke away at 
first motion. The resulting structures contained complex joints that would be difficult to 
design or manufacture using traditional methods (Fig 4d and Fig 5). Standard stepper 
motors were then snapped in, and the evolved neural network was executed on a 
microcontroller to activate the motors. The physical machines (3 to date) then faithfully 
reproduced their virtual ancestors’ behavior in reality (see Table I). 

In spite of the relatively simple task and environment (locomotion over an infinite 
horizontal plane), surprisingly different and elaborate solutions were evolved. Machines 
typically contained around 20 building blocks, sometimes with significant redundancy 
(perhaps to make mutation less likely to be catastrophic"). Not less surprising was the 
fact that some (e.g. Fig 5b) exhibited symmetry, which was neither specified nor 
rewarded for anywhere in the code; a possible explanation is that symmetric machines are 
more likely to move in a straight line, consequently covering a greater net distance and 
acquiring more fitness. Similarly, successful designs appear to be robust in the sense that 
changes to bar lengths would not significantly hamper their mobility. Three samples are 
shown and described in detail in Figure 5, exploiting principles of ratcheting (5a), anti- 
phase synchronization (5b) and dragging (5c). Others (not shown here) used a sort of a 
crawling bi-pedalism, where a body resting on the floor is advanced using alternating 
thrusts of left and right “limbs”. Some mechanisms use sliding articulated components to 
produce crab-like sideways motion. Other machines used a balancing mechanism to shift 
friction point from side to side and advance by oscillatory motion. Table I compares the 
performances of three physical creatures to their virtual ancestors. Note that although 


overall distance traveled in the 2" and 3™ cases does not match, in all cases the physical 


motion was achieved using corresponding mechanical and control implementations. The 
difference in distance results from slipping of the limbs on the surface, implying that the 
friction model used in the simulation was not realistic. 

In summary, while both the machines and task we describe in this work are fairly 
simple from the perspective of what human teams of engineers can produce, and what 
biological evolution has produced, we have demonstrated for the first time a robotic 
bootstrap, where automatically designed electromechanical systems have been 
manufactured robotically. We have carefully minimized human intervention both in the 
design and in the fabrication stages. Besides snapping in the motors, the only human 
work was in informing the simulation about the universe that could be manufactured. 

Without reference to specific organic chemistry, life is an autonomous design 
process that is in control of a complex set of chemical factories allowing the generation 
and testing of physical entities which exploit the properties of the medium of their own 
construction. Using a different medium, namely off-the-shelf rapid manufacturing, and 
evolutionary design in simulation, we have made progress towards replicating this 
autonomy of design and manufacture. This is the first time any artificial evolution system 
has been connected to an automatic physical construction system. All together, our 
evolutionary design system, solidification process, and rapid prototyping machine form a 
primitive “replicating” robot. While there are many, many further steps before this 
technology is dangerous”, we believe that if indeed artificial systems are to ultimately 
interact and integrate with reality, they cannot remain virtual; it is crucial that they cross 
the simulation-reality gap to learn, evolve”! and affect the physical world directly”. 
Eventually, the evolutionary process must accept feedback from the live performance of 


its products. 


Future work is primarily needed in understanding how more complex modular 
structures might self-organize, and how these complex structures may transfer into reality 


under control of the evolutionary process. Technological advances in MEMS, nano- 
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Fig. 5. Three resulting robots: (a) A tetrahedral mechanism that produces hinge-like motion 
and advances by pushing the central bar against the floor. (b) This surprisingly symmetric machine uses 
a 7-neuron network to drive the center actuator in perfect anti-phase with the two synchronized side 
limb actuators. While the upper two limbs push, the central body is retracted, and vice versa. (c) This 
mechanism has an elevated body, from which it pushes an actuator down directly onto the floor to 
create ratcheting motion. It has a few redundant bars dragged on the floor, which might be contributing 
to its stability. Print times are 22, 12 and 18 hours, respectively. These machines perform in reality is 
the same way they perform in simulation. Motion videos of these robots and others can be viewed at 
http://www.demo.cs.brandeis.edu. 


fabrication and multi-material rapid prototyping that can embed circuits” and actuators” 
in bulk structure, higher fidelity physical simulation, and increased understanding of 
evolutionary computational processes may pave the way for what Moravec has termed 


“Escape Velocity”. 


Methods 


Robot representation: A robot is represented by a string of integers and floating point 


numbers that describe bars, neurons, and their connectivity, as follows: 
robot := <vertices><bars><neurons><actuators> 
vertex := <X, Yy, Z> 
bar := <vertexl index, vertex2 index, relaxed length, stiffness> 
neuron := <threshold, synapse coefficients of connections to all neurons> 


actuator := <bar index, neuron index, bar range> 


Evolution process: Experiments were performed using version 1.2 of GOLEM 
(Genetically Organized Lifelike Electro Mechanics), which can be obtained from 
http://www.demo.cs.brandeis.edu/golem. We carried out a simulated evolutionary 
process: The fitness function was defined as the net Euclidean distance that the center-of- 
mass of an individual has moved over a fixed number (12) of cycles of its neural control. 
We started with a population of 200 null (empty) individuals. Each experiment used a 
different random seed. Individuals were then selected, mutated, and replaced into the 
population in steady-state as follows: The selection functions we tried were random, 
fitness proportionate or rank proportionate. The mutation operators used to generate an 
offspring were the following (with probability): Small mutation in length of bar or 
neuron synaptic weight (0.1), removal/addition of a small dangling bar or unconnected 
neuron (0.01), split vertex into two and add a small bar or split bar into two and add 
vertex (0.03), attach/detach neuron to bar (0.03). The dice were rolled until at least one 
mutation was applied. The mutations were applied directly to the symbolic representation 
of the phenotype. After mutation, a new fitness is assigned to the individual by means of 
a simulation of the mechanics and the control (see details below). The offspring was 


inserted into the population by replacing an existing individual. The replacement 
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functions we tried chose individuals to replace either randomly, in inverse-proportion to 
its fitness, or using similarity proportionate criteria (deterministic crowding”’). Various 
permutations of selection-replacement methods are possible; The results we report here 
were obtained using fitness proportionate selection and random replacement. However, 
using rank selection instead of fitness proportionate selection, or using random selection 
with fitness proportionate replacement yields equivalent results. . The process continued 
for 500-5000 generations (approx 10° to 10° evaluations overall). The process was carried 
out both serially and in parallel (on a 16-processor computer). On parallel computers we 
noticed an inherent bias towards simplicity: Simpler machines could complete their 
evaluation sooner and consequently reproduce more quickly than complex machines (this 
could be avoided with a generational implementation). 

Our evolutionary simulation was based on Evolutionary Srategies*’ and 
Evolutionary Programming”, since it directly manipulated continuous valued 
representations and used only elementary operators of mutation. Alternatively, we could 
have used Genetic Algorithms” and Genetic Programming”? that introduce cross-over 
operators sensitive to the structure of the machines, which might change the rate of 
evolution and lead to replicated structures. We did not form a morphological grammar 
from which the body is developed”', but evolved directly on the symbolic representation 
of the phenotype. And, instead of separating body (morphology) and brain (control) into 
separate populations, or providing for a “neonatal” stage that might allow us to select for 
brains that are able to learn to control their bodies, we simply applied selection to bodies 
and brains as integrated units. This simplified experimental setup followed our focus on 
completing the simulation and reality loop, but we anticipate that the many techniques 
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that have been developed in evolutionary and co-evolutionary learning will enrich 


our results. 


Simulation: Both the mechanics and the neural control of a machine were simulated 
concurrently. The mechanics were simulated using quasi-static motion, where each frame 
of the motion was assumed to be statically stable. This kind of motion is simple to 
simulate and easy to induce in reality, yet is rich enough to support various kinds of low- 


momentum motion like crawling and walking (but not jumping). The model consisted of 
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ball-joined cylindrical bars with true diameters. Each frame was solved by relaxation: An 
energy term was defined, taking into account elasticity of the bars, potential gravitational 
energy, and penetration energy of collision and contact. The degrees of freedom of the 
model (vertex coordinates) were then adjusted iteratively according to their derivatives to 
minimize the energy term, and the energy was recalculated. Static friction was also 
modeled. The use of relaxation permitted handling singularities (e.g. snap-through 
buckling) and under-constrained cases (like a dangling bar). Noise was added to ensure 
the system does not converge to unstable equilibrium points, and to cover the simulation- 
reality gap”. The material properties modeled correspond to the properties of the rapid 
prototyping material (E=0.896GPa, p=1000Kg/m* Oyicia=19MPa). The neural network 
was simulated in discrete cycles. In each cycle, actuator lengths were modified in small 


increments not larger than 1 cm. 
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